Papers with token-level correctness
SelFusion: Self-distillation for Diffusion Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods for autoregressive large language models (LLMs) are not effective for reducing generation quality, but they can be useful for real-time applications. |
| Approach: | They propose a self-distillation framework that allows for effective KD without external teacher . they propose to use two modes of knowledge distillation to determine distillation direction . |
| Outcome: | The proposed framework outperforms existing methods with external teachers on instruction-following tasks. |